Localization of a mobile agent, such as a portable device, a robot, or an autonomous vehicle, in various real-world environments is an important capability for smart navigation, location-based services, and mixed reality (MR) experiences. Conventional localization techniques largely employ visual features such as SIFT, ORB, or DAISY to represent a scene and localize a mobile sensor in a scene or camera centered coordinate system. Such techniques include use of various flavors of SLAM (simultaneous localization and mapping) that have been implemented on smartphones, virtual reality (VR)/augmented reality (AR)/MR devices, and autonomous vehicles.
A fundamental problem with such feature-based methods includes the variance and instability of computed features for scenes encountered at different times of day, different times of year, different lighting conditions, changes in viewpoint, and/or features located on moveable or moving objects. Conventional localization techniques fail to provide robustness to such variations, which typically occur in common real-world environments. As a result, building effective and stable visual maps of the world that are robust in facilitating localization of mobile agents remains an unfulfilled dream.